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This is in response to the appeal brief filed 10/26/2007 appealing from the Office action mailed 
04/30/2007. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings 
which will directly affect or be directly affected by or have a bearing on the Board's decision in 
the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection contained in 
the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 
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(8) Evidence Relied Upon 

5,9 15,250 Jainetal. 6-1999 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claim Rejections - 35 USC §102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

5. Claims 1, and 3-28 are rejected under 35 U.S.C. 102(b) as being anticipated by Jain et al. 
U.S. Patent No. 5,915,250. 

Regarding claim 1, Jain teaches a computer implemented method (FIG. 1 A) for 
determining the semantic meaning of mages, comprising: 

deriving a set of perceptual semantic categories (defining and register custom primitives) 
(abstract) for representing important semantic cues (object's attributes) (abstract) (color, 
texture, shape, pattern, object) (face) (column 4, lines 1-45) in the human perception of images 
(column 4, lines 20-25), where each semantic category is modeled through a combination of 
perceptual features that define the semantics of that category and that discriminate that category 
from other categories (visual information retrieval system provides modular and framework to 
define and classify category of object's attributes) (abstract and column 3, lines 60 to column 4, 
lines 1-10), wherein the perceptual features and their combinations are derived at least in part 
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through subjective experiments performed with human observers (column 4, lines 21-32; FIG. 
1A, elements 102, 104 and 1 12; column 6, lines 58-61; column 8, lines 32-35; column 11, lines 
43-59 and column 18, lines 7-26); and 

for each semantic category, forming a set of the perceptual features as a complete feature 
set CFS (feature vector to store each kind of primitives) (column 5, lines 5-20). 

For claim 3, Jain further teaches a method further comprising extracting perceptual 
features from an input image and applying a perceptually-based metric (a similarity scoring 
system) to determine the semantic category for that image (abstract). 

Referring to claim 4, Jain discloses a method comprising processing the input image to 
compute the CFS (FIG. 1*A, element 1 12); comparing the input image to each semantic category 
through the perceptually-based metric that computes a similarity measure between the features 
used to describe the semantic category and the corresponding features extracted (abstract) from 
the input image (rank similarity result) (column 12, lines 50-67); and assigning the input image 
to the semantic category that corresponds to a highest value of the similarity measure (column 
12, lines 65 to column 13, line 3). 

For claims 5, Jain also discloses a method further comprising computing features from 
the CFS for images in an image database (FIG. 1 A); and generating a distance measure for 
characterizing a relationship of a selected image to another image in the image database by 
applying a perceptually-based similarity metric (column 8, lines 10-27). 

For claim 6, Jain shows a method where values of the similarity metric computed for 
images in the image database are subsequently used to search for similar images 
in the image database (column 9, lines 50-64). 
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Regarding claim 7, Jain shows a method where values of the similarity metric computed 
for images in the image database are subsequently used to organize images in the 
image database (classifying images base on similarity scoring system) (FIG. 5B, elements 280 
and 284). 

For claim 8, Jain teaches a method where values of the similarity metric computed for 
images in the image database are subsequently used to display images in the image database in 
an organized manner (FIG. 4; FIG. 5B, element 294; and column 1 1, lines 23-39). 

Also to claim 9, Jain further teaches a method further comprising defining a subset of 
features for the selected image or for an image retrieved from the image database, and using the 
subset of features to refine a search through the image database (searching with specific 
parameters) (column 4, lines 29-45). 

Referring to claim 10, Jain shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network (FIG. IB and column 
9, lines 10-25)., 

Also to claim 11, Jain further shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the step of 
characterizing the relationship of the selected image to another image in the image database by 
applying the perceptually-based similarity metric is accomplished to retrieve an image from the 
remote image database (FIG. IB and column 9, lines 10-50). 

As for claim 12, Jain also teaches a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the 
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step of characterizing the relationship of the selected image to another image in the image 
database by applying the perceptually-based similarity metric is accomplished in conjunction 
with a text-based search algorithm to retrieve a multi-media object from the remote location 
(direction communication between databases) (column 9, lines 10-50). 

Referring to claim 13, Jain discloses a method wherein to assign a particular semantic 
category to an image all of a set of Required Features must be present in the image, and at least 
one of a set of Frequently Occurring Features must be present in the image (fixed/universal or 
default primitives) (column 8, line 60 and column 16, lines 16-25). 

For claim 14, please refer back to claims 1-3 for further teachings and explanations. In 
addition, Jain teaches a data processing system comprising a data processor, a graphical user 
interface and a memory to processes the aforementioned limitations (FIG. 1 A and column 9, 
lines 5-50). 

For claims 15-24, please refer back to claims 4-13 respectively for further teachings and 
explanations. 

For claim 25, please refer back to claims 1-4 for further teachings and explanations. In 
addition, Jain teaches a computer program (column 6, lines 15-25) to processes aforementioned 
limitation and semantic categories being modeled using multidimensional scaling and 
hierarchical clustering techniques (different ways of querying and classifying images) (FIG. 1 A, 
elements 106 and 108). 

For claims 26 - 27, Jain teaches a computer program where as a result of comparing the 
input image to images stored in the image database one or more most similar 
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images are identified in the image database (column 12, lines 50-67 and (column 12, lines 65 to 
column 13, line 3) and display it (FIG. IB, element 148). 

. For claim 28, please refer back to claim 14 for further teachings and explanations. 
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10) Response to Argument 

(A) The following discussion relates to the rejection of claims 1 and 3-28 as being 
unpatentable under 35 U.S.C. 102(b) as being anticipated by Jain et al. U.S. Patent No. 5,915,250 
(herein Jain). 

1 . Appellant' s Argument The Appellant argues (pages 8-11 of the Appeal Brief) 

that Jain does not teach "the perceptual features and their combinations are derived at least in 
part through subjective experiments performed with human observers". 

Examiner's Response - — The Examiner respectfully disagrees. To better explain 



the Examiner's ground of rejection, a table's presentation and FIG. 6 of U.S. Patent No. 
5,915,250 to Jain et al. ("Jain") are provided below to show rejections of all the features in each 
limitation of the claim 1 : 



Limitations 


Art Rejection's Citations (Jain) and 
explanations 


1 . "Deriving a set of perceptual semantic 
categories" 


The Examiner interprets the derivation of 
primitives such as "local color", "global color", 
texture ...etc. as shown in FIG. 6-see below, 
"PRIMTIVES" and column 4, lines 1-10. 


2. "for representing information semantic 
cues" 

<<>■ 


Representing information of the image such as 
"semantic-rich representation of visually 
salient characteristics". See column 4, lines 10- 
15. 
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3. "in the human perception of images" 


Visual information by users of images. See 
Abstract, lines 1-5; and column 3, lines 20-25; 


4. "where each semantic category (as 
discussed in the table 1.) is modeled through a 
combination of perceptual features that define 
the semantics to that category. 


Query primitives into features vectors by 
different metrics such as weighted 
contributions. See column 8, lines 10-35. 


5. "and that discriminate that category from 
other categories" 


Generate different topological spaces for 
primitives. See column 8, lines 9-11. 


6. "wherein the perceptual features" 


Color, texture, shape and motion 
characteristics. See column 3, lines 9-11. 


7. "and their combinations are derived at least 
in part" (emphasis added). 


The analysis of similarity with respect to visual 
attributes. See column 6, lines 20-26. 


8. "through subjective experiments performed 
with human observers;" 


Column 4, lines 19-40 and column 8, lines 24- 
35. 


9. "For each semantic category, forming a set 
of the perceptual features a complete feature 
setCFS" 


Base Visual Information Retrieval (VIR) 
engine and Extensible Engine provide fixed set 
of primitives and installing new primitives for 
the VIR engine to update a complete feature 
set. See column 8, lines 58-63 and column 20, 1 
lines 30-40. 



Application/Control Number: 

10/033,597 

Art Unit: 2624 



Page 10 



MS. Patent 



Juiu 22, IW 



5,915,250 




Thus, it is clear that Jain teaches "the perceptual features (Color, texture, shape and 
motion characteristics) (See column 3, lines 9-1 1) and their combinations are derived at least in 
part (The analysis of similarity with respect to visual attributes) (See column 6, lines 20-26) 
through subjective experiments performed with human observers (Column 4, lines 19-40 and 
column 8, lines 24-35)". 
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2. Appellant's Argument — Regarding claims 4 and 1 5, the Appellant argues 
(pages 14 of the Appeal Brief) that Jain does not teach "assigns the input image to the semantic 
category that corresponds to a highest value of the similarity measure". 

Examiner's Response — The Examiner respectfully disagrees. Jain teaches a 
concept of assign (primitives of image are ordered by weights) (column 24, lines 25-30) highest 
weighted primitive ordered first for in the similarity analysis when processing images (column 
24, lines 21-64). 

3. Appellant's Argument — Regarding claims 6 and 17, the Appellant argues 
(pages 17 of the Appeal Brief) that Jain does not teach a "similarity metric" is "used to search for 
similar images in the image database". 

Examiner's Response — The Examiner respectfully disagrees. Clearly, a 
concept of similarity score is a similarity metric can be used to search for similar images in an 
image database. See column 9, lines 50-60. Also, it is disclosed by Jain that similarity is a 
metric in the art (column 3, lines 50-52). In addition, other metrics are used by Jain are "distance 
metrics" (column 8, line 8), and topological metrics (column 8, line 27). 

4. Appellant's Argument — Regarding claims 8, 19 and 27, the Appellant argues 
(pages 1 8 of the Appeal Brief) that Jain does not teach "a method where values of the similarity 
metric computed for images in the image database are subsequently used to display images in the 
image database in an organized manner" or similar in scope. 

Examiner's Response — The Examiner respectfully disagrees. Jain teaches a 
method where values of the similarity metric (similarity score) (column 9, lines 50-60) computed 
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for images in the image database (column 9, lines 17-40) are subsequently used to display 
images in the image database in an organized manner (column 9, lines 60-64). 

5. Appellant's Argument - — Regarding claims 9 and 20, the Appellant argues 
(pages 19 of the Appeal Brief) that Jain does not teach "a method comprising defining a subset 
of features for the selected image or for an image retrieved from the image database, and using 
the subject of features to refine a search through the image database" or similar in scope. 

Examiner's Response — The Examiner respectfully disagrees. Jain teaches a 
method comprising defining a subset of features for the selected image (specify a property or 
attribute of the image) (column 4, lines 20-25) or for an image retrieved from the image 
database, and using the subject of features to refine a search through the image database (query 
search refinement for browsing) (column 4, lines 33-38). 

5. Appellant's Argument — - Regarding claims 1 0 and 2 1 , the Appellant argues 
(pages 20 of the Appeal Brief) that Jain does not teach "a system wherein the image database is 
located at a remote location and is reachable through a data communications network that is 
bidirectionally coupled to said data processor through a network interface" or similar in scope. 

Examiner's Response — The Examiner respectfully disagrees. Jain teaches a 
system wherein the image database (can be either storage, backup unit or server with backup 
facility) (FIG. IB) is located at a remote location and is reachable through a data 
communications network that is bidirectionally coupled (double headed arrow shows the 
bidirectionally communication) (FIG. IB) to said data processor through a network interface 
(network) (FIG. IB, element 166). More readings can be found at column 9, lines 10-50. 
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6. Appellant's Argument — Regarding claims 12 and 23, the Appellant argues 
(pages 21 of the Appeal Brief) that Jain does not teach "a system where the data processor 
applies the perceptually-based similarity metric to in conjunction with a text-based search 
algorithm to retrieve a multi-media object from the remote location" or similar in scope. 

Examiner's Response — The Examiner respectfully disagrees. Jain teaches a 
system where the data processor (FIG. I B, element 144) applies the perceptually-based similarity 
metric (similarity score) (column 6, lines 25-67) to in conjunction with a text-based search 
algorithm to retrieve a multi-media object from the remote location (textual query attributes such 
as Keywords to retrieve images or image objects) (column 11, lines 11-21). 

7. Appellant's Argument — Regarding claims 13, 24 and 28, the Appellant argues 
(pages 22 of the Appeal Brief) that Jain does not teach "a method wherein to assign a particular 
semantic category to an image all of a set of Required Features must be present in the image, and 
at least one of a set of Frequently Occurring Features must be present in the image" or similar in 
scope. 

Examiner's Response - — The Examiner respectfully disagrees. Jain teaches a 
method wherein to assign a particular semantic category to an image all of a set of Required 
Features must be present in the image ("VIR Engine has a stateless architecture in which all of 
the data about images is managed and stored by the application") (column 6, lines 50-53), and at 
least one of a set of Frequently Occurring Features must be present in the image (a primitive 
such as color, shape or texture always exist in the image) (FIG. 6, elements 330, 332, 334, 336 
and 338; column 4, lines 5-10). 
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8. 



Appellant's Argument — - Regarding claim 26, the Appellant argues (pages 24 of 



the Appeal Brief) that Jain does not teach the computer program of claim 25 and thus claim 26 is 
not anticipated by Jain. 



teaches a computer program (application modules) (FIG. 1 A; column 6, lines 42-44 and column 
28, lines 30-40). 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related 
Appeals and Interferences section of this examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 
Brian Le 

Primary Examiner 



Examiner's Response - — The Examiner respectfully disagrees. Jain clearly 




Conferees: 



Matthew Bella 




MATTHEW C. BELLA 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 
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